<!--
Google IO 2012 HTML5 Slide Template

Authors: Eric Bidelman <ebidel@gmail.com>
         Luke Mahé <lukem@google.com>

URL: https://code.google.com/p/io-2012-slides
-->
<!DOCTYPE html>
<html>
<head>
  <title>What would you do with unlimited computing resources?</title>
  <meta charset="utf-8">
  <meta http-equiv="X-UA-Compatible" content="chrome=1">
  <!--<meta name="viewport" content="width=device-width, initial-scale=1.0, minimum-scale=1.0">-->
  <!--<meta name="viewport" content="width=device-width, initial-scale=1.0">-->
  <!--This one seems to work all the time, but really small on ipad-->
  <!--<meta name="viewport" content="initial-scale=0.4">-->
  <meta name="apple-mobile-web-app-capable" content="yes">
  <link rel="stylesheet" media="all" href="theme/css/default.css">
  <link rel="stylesheet" media="only screen and (max-device-width: 480px)" href="theme/css/phone.css">
  <base target="_blank"> <!-- This amazingness opens all links in a new tab. -->
  <script data-main="js/slides" src="js/require-1.0.8.min.js"></script>
</head>
<body style="opacity: 0">

<slides class="layout-widescreen">

  <slide class="logoslide nobackground">
    <article class="flexbox vcenter">
      <span><img src="images/google_developers_logo.png"></span>
    </article>
  </slide>

  <slide class="title-slide seque nobackground">
    <aside class="gdbar"><img src="images/google_developers_icon_128.png"></aside>
    <!-- The content of this hgroup is replaced programmatically through the slide_config.json. -->
    <hgroup class="auto-fadein">
      <br>&nbsp;<br>
      <br>&nbsp;<br>
      <br>&nbsp;<br>
      <br>&nbsp;<br>
      <h1 data-config-title><!-- populated from slide_config.json --></h1>
      <h2 data-config-subtitle><!-- populated from slide_config.json --></h2>
      <p data-config-presenter><!-- populated from slide_config.json --></p>
    </hgroup>
  </slide>

  <slide>
    <hgroup>
      <h2>The last thirty years...</h2>
    </hgroup>
    <article>
      <table>
        <tr>
          <th>Attribute</th><th>1985 Mainframe</th><th>2012 Rack Server</th><th>Improvement</th>
        </tr>
        <tr>
          <td>CPU</td><td>60 MHz</td><td>2 GHz</td><td class="highlight">33X</td>
        </tr>
        <tr>
          <td>RAM</td><td>256 MB</td><td>1 TB</td><td class="highlight">4000X</td>
        </tr>
        <tr>
          <td>Disk</td><td>3 GB</td><td>6 TB</td><td class="highlight">2000X</td>
        </tr>
        <tr>
          <td>Cost</td><td>$5 M</td><td>$2.5 K</td><td class="highlight">2000X</td>
        </tr>
      </table>
      <br>&nbsp;</br>
      <div class="build" style="font-size: xx-large;">
        <span>Compared to a $5M computer in 1985, my smart phone...</span>
        <span>is 20x faster,</span>
        <span>has 4 times the RAM & disk capacity,</span>
        <span>costs 10,000x less,</span>
        <span>and fits in my pocket.</span>
      </div>
    </article>
  </slide>

  <slide class="fill white" style="background-image: url(images/dog.jpg);">
    <hgroup>
      <h2 class="white">But we still want more...</h2>
    </hgroup>
    <article>
      <ul class="build" style="font-size: xx-large;">
        <li>Demand expands to consume available resources.
          <ul>
            <li>Solving big problems still requires big $</li>
            <li>Hardware becomes obsolete quickly</li>
          </ul>
        </li>
      </ul>
      <br>&nbsp;
      <ul class="build" style="font-size: xx-large;">
        <li>Cloud computing is the answer!</li>
      </ul>
      <footer class="source">source: <a href="http://www.flickr.com/photos/sparker/3372002117/">http://www.flickr.com/photos/sparker/3372002117/</a></footer>
    </article>
  </slide>

  <slide class="fill" style="background-size: 100%; background-image: url(images/samuel.jpeg);">
  </slide>

  <slide class="fill" style="background-image: url(images/cloud.jpg);">
    <hgroup>
      <h2>Google's History in the Cloud</h2>
    </hgroup>
    <article class="black">
      <ul style="font-size: 48;">
        <li>Google Search - 1998</li>
        <li>Gmail - 2004</li>
        <li>Google Docs - 2005</li>
        <li>Google Maps - 2005</li>
        <li>YouTube - 2005</li>
        <li>Google Apps - 2006</li>
        <li>Google App Engine - 2008</li>
        <li>Google Cloud Storage - 2010</li>
        <li>Google Compute Engine - 2012</li>
      </ul>
      All of these products are, essentially, massively scalable, highly secure, ultra-reliable cloud computing services built on Google's infrastructure.
      <footer class="source">source: <a href="http://www.flickr.com/photos/supertin/1368972764/">http://www.flickr.com/photos/supertin/1368972764/</a></footer>
    </article>
  </slide>

  <slide>
    <hgroup>
      <h2>Not just any infrastructure<h2>
    </hgroup>
    <article>
      <p>Unparalleled scale, speed and global reach.
      <p>
      <ul class="build">
        <li>over 72 hours of video uploaded every minute
        <li>more than 100 million gigabytes (100 petabytes) indexed
        <li>425 million active users
        <li>250 ms average response time on billions of search per day
        <li>global, secure, high performance private network
      </ul>
    </article>
  </slide>

  <slide>
    <hgroup>
      <h2>Google Compute Engine</h2>
    </hgroup>
    <article>
      <ul class="build">
        <li>Infrastructure as a Service (Iaas)</li>
        <li>Virtual Machines running in Google's infrastructure</li>
        <li>Leverages Google's rich experience building cloud services</li>
        <li>Advanced networking, performance, scalability, and security features</li>
        <li>Plays well with other parts of the <a href="http://cloud.google.com/">Google Cloud Platform</a>
          <ul>
            <li>App Engine</li>
            <li>Google Cloud Storage</li>
            <li>Google Big Query</li>
            <li>Google Cloud SQL</li>
            <li>Google Prediction API</li>
            <li>Google Translation API</li>
          </ul>
        </li>
      </ul>
    </article>
  </slide>

  <slide class="segue dark nobackground">
    <aside class="gdbar"><img src="images/google_developers_icon_128.png"></aside>
    <hgroup class="auto-fadein">
      <h2>Web UI and Command Line Demo</h2>
      <h3>Google API Console and gcutil</h3>
    </hgroup>
  </slide>

  <slide>
    <hgroup>
      <h2>Compute Engine Overview</h2>
    </hgroup>
    <article>
      <p style="float: right;">
        <img src="images/overview.png" style="height: 500px;">
      </p>
      <ul>
        <li>Project</li>
        <li>Instance</li>
        <li>Network</li>
        <li>Storage</li>
        <li>API &amp; Tools</li>
      </ul>
    </article>
  </slide>

  <slide>
    <hgroup>
      <h2>Project</h2>
    </hgroup>
    <article>
      <p style="float: right;">
        <img src="images/project.png" style="height: 400px;">
      </p>
      <ul>
        <li>Based on API Console projects</li>
        <li>Container for all resources</li>
        <li>Team membership</li>
        <li>Billing</li>
      </ul>
    </article>
  </slide>

  <slide>
    <hgroup>
      <h2>Instance</h2>
    </hgroup>
    <article>
      <p style="float: right;">
        <img src="images/instances.png" style="height: 400px;">
      </p>
      <p>
        Linux Virtual Machines
      </p>
      <ul>
        <li>Root access</li>
        <li>Stock images: Ubuntu, CentOS</li>
        <li>Modern processor<br>(Intel Sandy Bridge Xeon)</li>
        <li>1, 2, 4, and 8 virtual CPUs</li>
        <li>3.75GB RAM per virtual CPU</li>
      </ul>
    </article>
  </slide>

  <slide>
    <hgroup>
      <h2>Network</h2>
    </hgroup>
    <article>
      <p style="float: right;">
        <img src="images/network_private.png" style="height: 400px;">
      </p>
      <p>
        Virtual private network
      </p>
      <ul>
        <li>Isolated network per project</li>
        <li>Private IPv4 space</li>
        <li>IP level (Layer 3) network</li>
        <li>Internal facing DNS</li>
      </ul>
    </article>
  </slide>

  <slide>
    <hgroup>
      <h2>Network</h2>
    </hgroup>
    <article>
      <p style="float: right;">
        <img src="images/network_internet.png" style="height: 400px;">
      </p>
      <p>
        Internet access
      </p>
      <ul>
        <li>External IPs</li>
        <li>1-to-1 NAT</li>
        <li>Built in ﬁrewall system</li>
      </ul>
    </article>
  </slide>

  <slide>
    <hgroup>
      <h2>Storage</h2>
    </hgroup>
    <article>
      <p style="float: right;">
        <img src="images/storage_pd.png" style="height: 400px;">
      </p>
      <p>
        Persistent disk
      </p>
      <ul>
        <li>Provisioned via API</li>
        <li>Located in same zone</li>
        <li>Encrypted at rest</li>
      </ul>
    </article>
  </slide>

  <slide>
    <hgroup>
      <h2>Storage</h2>
    </hgroup>
    <article>
      <p style="float: right;">
        <img src="images/storage_local.png" style="height: 400px;">
      </p>
      <p>
        Ephemeral disk
      </p>
      <ul>
        <li>Lives and dies with instance</li>
        <li>On the same physical machine</li>
        <li>Encrypted at rest</li>
      </ul>
    </article>
  </slide>

  <slide>
    <hgroup>
      <h2>Storage</h2>
    </hgroup>
    <article>
      <p style="float: right;">
        <img src="images/storage_cloud.png" style="height: 400px;">
      </p>
      <p>
        Cloud storage
      </p>
      <ul>
        <li>Internet object store</li>
        <li>Global API based access</li>
        <li>Frictionless access with service accounts</li>
      </ul>
    </article>
  </slide>

  <slide>
    <hgroup>
      <h2>Tools</h2>
    </hgroup>
    <article>
      <p style="float: right;">
        <img src="images/tools.png" style="height: 400px;">
      </p>
      <ul>
        <li>Google Client Libraries</li>
        <li>gcelib</li>
        <li>Command line tool - gcutil</li>
        <li>Compute Engine UI</li>
        <li>Partners and ecosystem</li>
      </ul>
    </article>
  </slide>

  <slide>
    <hgroup>
      <h2>gcutil</h2>
    </hgroup>
    <article>
      <p>
        Start a new instance
      </p>
      <ul><li><code>
        gcutil addinstance my-instance
      </code></li></ul>
      <p>
        List all running instances
      </p>
      <ul></li><code>
        gcutil listinstances --filter="status eq RUNNING"
      </code></li></ul>
      <p>
        Add a network called prod
      </p>
      <ul><li><code>
        gcutil addnetwork prod
      </code></li></ul>
      <p>
        Setup a firewall to accept incoming http connections on port 80
      </p>
      <ul><li><code>
        gcutil addfirewall web --network=prod --allowed="tcp:http"
      </code></li></ul>
    </article>
  </slide>

  <slide>
    <hgroup>
      <h2>Compute Engine API</h2>
    </hgroup>
    <article>
      <p>
        REST API
      </p>
      <pre class="prettyprint">
        POST https://www.googleapis.com/compute/v1beta12/projects/project/instances
        {
          "name": string,
          "image": string,
          "machineType": string,
          "status": string,
          "statusMessage": string,
          "zone": string,
          "networkInterfaces": [...],
          "disks": [...],
          "metadata": {...},
          "serviceAccounts": [...]
        }
      </pre>
    </article>
  </slide>

  <slide class="segue dark nobackground">
    <aside class="gdbar"><img src="images/google_developers_icon_128.png"></aside>
    <hgroup class="auto-fadein">
      <h2>Google Compute Engine API Demo</h2>
    </hgroup>
  </slide>

  <slide>
    <hgroup>
      <h2>API Demo Code Highlights</h2>
    </hgroup>
    <article>
      <p>App Engine Front End Dispatches Requests to Back Ends</p>
      <pre class="prettyprint">
  def post(self):
    user_id = users.get_current_user().user_id()
    num_instances = int(self.request.get('num_instances'))
    rows = int(math.sqrt(num_instances))
    zoom = ZOOM_MAP[num_instances]
    q = taskqueue.Queue('default')
    task_list = []
    <b>for i in range(rows):
      for j in range(rows):
        task = taskqueue.Task(url='/%s/add' % (DEMO_NAME), 
            params={ 'user_id': user_id, 'zoom': zoom, 'x': i, 'y': j })
        task_list.append(task)
    q.add(task_list)</b>
      </pre>
    </article>
  </slide>

  <slide>
    <hgroup>
      <h2>API Demo Code Highlights</h2>
    </hgroup>
    <article>
      <p>App Engine Back End Services Requests (Creating Instances)</p>
      <pre class="prettyprint">
  def post(self):
    user_id = self.request.get('user_id')
    zoom = self.request.get('zoom')
    x = self.request.get('x')
    y = self.request.get('y')
    credentials = oauth2storage.StorageByKeyName(
        oauth2storage.CredentialsDatastore, user_id, 'credentials').get()
    api = gce.Gce(SERVICE, credentials, PROJECT_ID, ZONE, MACHINE_TYPE, DISK,
                  IMAGE, SERVICE_EMAIL, TAG, startup_script=STARTUP_SCRIPT)
    metadata = [
      { 'key': 'zoom', 'value': zoom }, 
      { 'key': 'x', 'value': x }, 
      { 'key': 'y', 'value': y }]
    <b>results = api.start_instance('%s-%s-%s' % (TAG, x, y), metadata)</b>
      </pre>
    </article>
  </slide>

  <slide>
    <hgroup>
      <h2>API Demo Code Highlights</h2>
    </hgroup>
    <article>
      <p>Wrapper Class Abstracting Python Client Library</p>
      <pre class="prettyprint">
def start_instance(self, instance_name):
  auth_http = get_auth_http(self.credentials)
  logging.info('Starting %s' % (instance_name))
  instance = {
      'name': instance_name,
      'zone': '%s/zones/%s' % (self.project_url, self.zone),
      'networkInterfaces': [{'network': '%s/networks/%s' % (self.project_url, self.network), ... }],
      'serviceAccounts': [{'scopes': [ CS_SCOPE ], ... }],
      ...
  }
  <b>request = self.service.instances().insert(project=self.project_id, body=instance)
  response = request.execute(auth_http)</b>
  return response
      </pre>
    </article>
  </slide>

  <slide>
    <hgroup>
      <h2>API Demo Code Highlights</h2>
    </hgroup>
    <article>
      <p>App Engine Front End Notifies Instances to Generate Tiles</p>
      <pre class="prettyprint">
class NotifyInstances(webapp2.RequestHandler):
  """Upload the 'go' file to Cloud Storage."""

  def post(self):
    user = users.get_current_user()
    credentials = oauth2storage.StorageByKeyName(
        oauth2storage.CredentialsDatastore, user.user_id(), 'credentials').get()
    <b>upload_results = cs.Cs().upload(
        credentials.access_token, CS_BUCKET, 'go/-', '-')</b>
    logging.info(upload_results)
      </pre>
    </article>
  </slide>

  <slide>
    <hgroup>
      <h2>MapR Hadoop Terasort Record Attempt</h2>
    </hgroup>
    <article>
      <table>
        <tr>
          <th></th><th>World Record</th><th>MapR on Google Compute Engine</th>
        </tr>
        <tr>
          <td>Time to sort</td><td>1:02</td><td>1:20</td>
        </tr>
        <tr>
          <td>Number of Servers</td><td>1,460</td><td>1,256</td>
        </tr>
        <tr>
          <td>Number of Cores</td><td>11,680</td><td>5,024</td>
        </tr>
        <tr>
          <td>Time to build cluster</td><td class="highlight">Months</td><td class="highlight">Minutes</td>
        </tr>
        <tr>
          <td>Capital Expenditure</td><td class="highlight">$5,840,000</td><td class="highlight">$728 (per hour, could run 45 times!)</td>
        </tr>
      </table>
    </article>
  </slide>

  <slide class="segue dark quote nobackground">
    <aside class="gdbar right bottom"><img src="images/google_developers_icon_128.png"></aside>
    <article class="flexbox vleft auto-fadein">
      <q>
      With Google Compute Engine, we were blown away. 
      We could not believe that it was a virtualized environment -- this 
      felt even better than a real environment. 
      </q>
      <div class="author">
        M.C. Srivas<br>
        CTO & Co-Founder, <a href="http://www.mapr.com/">MapR Technologies</a>
      </div>
    </article>
  </slide>

  <slide class="segue dark nobackground">
    <aside class="gdbar"><img src="images/google_developers_icon_128.png"></aside>
    <hgroup class="auto-fadein">
      <h2>Let's make a movie!</h2>
    </hgroup>
  </slide>

  <slide>
    <hgroup>
      <h2>Cluster Design</h2>
    </hgroup>
    <article>
      <img src="images/demo.png">
      <br>&nbsp;<br>
      <ul>
        <li>This takes about 8 hours on my Linux workstation</li>
        <li>or a few minutes on my 1K node Google Compute Engine cluster.</li>
      </ul>
    </article>
  </slide>

  <slide>
    <hgroup>
      <h2>Creating the master and the slaves</h2>
    </hgroup>
    <article>
      <pre class="prettyprint">
  echo creating master
  gcutil addinstance master --image=master --service_account_scopes=$SCOPES
    --zone=$ZONE --machine_type=$MACHINE_TYPE

  for i in $(eval echo {$START..$STOP})
  do
    NAME="s$i"
    echo creating slave $NAME
    <b>gcutil addinstance $NAME --image=slave --metadata_from_file=startup-script:startup
      --service_account_scopes=$SCOPES --zone=$ZONE --machine_type=$MACHINE_TYPE</b>
  done
      </pre>
    </article>
  </slide>

  <slide>
    <hgroup>
      <h2>The slave logic</h2>
    </hgroup>
    <article>
      <pre class="prettyprint">
FRAME=`uname -n | cut -c2-`
while true
do
  GO=`ls /mnt/master | grep "^go$" | wc -l`
  if [ "$GO" = 1 ]
  then
    echo clearing old frames...
    rm -f *.png
    echo obtaining local copy of animation sequence file...
    gsutil cp gs://marc-gce/seq.flam3 .
    <b>echo generating animation frame...
    env begin=$FRAME end=$FRAME flam3-animate &lt;seq.flam3</b>
    echo copying generated frames to master...
    cp *.png /mnt/master/frames
  fi
  sleep 1
done
</pre>
    </article>
  </slide>

  <slide>
    <hgroup>
      <h2>Here's our video...</h2>
    </hgroup>
    <article>
      <a href="http://youtu.be/zVZtCSeiVj4?rel=0">http://youtu.be/zVZtCSeiVj4</a>
      <footer class="source">
        frame generation software: <a href="http://code.google.com/p/flam3/">http://code.google.com/p/flam3/</a><br>
        audio track: <a href="http://soundcloud.com/chuckadile/clocks-coldplay-cover-instrumental">http://soundcloud.com/chuckadile/clocks-coldplay-cover-instrumental</a>
      </footer>
    </article>
  </slide>

  <slide>
    <hgroup>
      <h2>Limited Preview</h2>
    </hgroup>
    <article>
      <ul>
        <li>This product is now serving production Google traffic
        <li>The preview is focused on computation and/or I/O intensive batch workloads</li>
        <li>Complimentary access for a limited period</li>
        <li>SLA and support available to commercial customers</li>
        <li>Apply for program at <a href="http://cloud.google.com">http://cloud.google.com</a></li>
 
      </ul>
    </article>
  </slide>

  <slide class="fill" style="background-image: url(images/cube.jpg)">
    <article>
      <h3>Fast,<br>scalable,<br>consistent,<br>reliable,<br>secure,<br>proven IaaS:<br><b>not so easy.</b></h3>
      <br>
    </article>
  </slide>

  <slide>
    <hgroup>
      <h2>Where to learn more</h2>
    </hgroup>
    <article>
      <ul>
        <li>Google Cloud Platform</li>
        <ul><li><a href="http://cloud.google.com/compute">http://cloud.google.com/compute</a></li></ul>
        <li>Google Compute Engine Documentation</li>
        <ul><li><a href="http://developers.google.com/compute">http://developers.google.com/compute</a></li></ul>
        <li>Google Compute Engine Announcement Group</li>
        <ul><li><a href="https://groups.google.com/forum/?fromgroups#!forum/gce-announce">https://groups.google.com/forum/?fromgroups#!forum/gce-announce</a></li></ul>
        <li>Google Compute Engine Discussion Group</li>
        <ul><li><a href="https://groups.google.com/forum/?fromgroups#!forum/gce-discussion">https://groups.google.com/forum/?fromgroups#!forum/gce-discussion</a></li></ul>
      </ul>
    </article>
  </slide>

  <slide class="fill nobackground" style="background-image: url(images/horizon.jpeg)">
    <hgroup>
      <h2>So...what would you do with unlimited<br>computing resources?</h2>
    </hgroup>
    <article class="black" style="font-size: x-large;">
      <footer class="source">source: Matthew Leske, Google</a></footer>
    </article>
  </slide>

  <slide class="thank-you-slide segue nobackground">
    <aside class="gdbar right"><img src="images/google_developers_icon_128.png"></aside>
    <article class="flexbox vleft auto-fadein">
      <h2>&lt;Thank You!&gt;</h2>
    </article>
    <p class="auto-fadein" data-config-contact>
      <!-- populated from slide_config.json -->
    </p>
  </slide>

  <slide class="logoslide dark nobackground">
    <article class="flexbox vcenter">
      <span><img src="images/google_developers_logo_white.png"></span>
    </article>
  </slide>

  <slide class="backdrop"></slide>

</slides>

<script>
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-XXXXXXXX-1']);
_gaq.push(['_trackPageview']);

(function() {
  var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
  ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
  var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script>

<!--[if IE]>
  <script src="http://ajax.googleapis.com/ajax/libs/chrome-frame/1/CFInstall.min.js"></script>
  <script>CFInstall.check({mode: 'overlay'});</script>
<![endif]-->
</body>
</html>
